Software developers use various mechanisms to enhance the quality of source code during software development. Such mechanisms can add tremendous value by improving quality and user experience as well as by reducing the need for post-release patches. However, such mechanisms can be costly and constrained.
Developers often aim to reach a certain code quality. Source code quality can be measured on attributes such as performance, compliance with expected functionality, robustness, complexity, efficiency, portability, usability of the source code, etc. In general, failures that occur during testing or during use of a software application serve as indicators of how well the application complies with its expected functionality (i.e., does the application function correctly or not?).
A particular approach to finding failures in a software application includes generation of so-called “test cases”. Once generated, a set of test cases can be executed to test sections of source code for compliance with expected functionality. A common definition for a test case is a set of conditions or variables under which a tester will determine if a requirement or use case upon an application is partially or fully satisfied. Sometimes it takes many test cases to determine that a requirement is fully satisfied. Thus, a test case can be characterized by a set of conditions which help to determine whether a requirement upon an application is satisfied or not.
In a test case approach to quality, test prioritization helps to determine a test execution order or an order of areas for test case development. For example, some tests may be more pervasive than others and hence results from a pervasive test may be dispositive, i.e., eliminate the need for one or more “lower priority” tests. In the latter instance, test prioritization may simply decide to develop test cases for highly traveled paths. While no particular logic exists as a standard for test prioritization, test prioritization nevertheless can lower costs by raising quality confidence in an efficient way.
Once some source code is tested (or marked for testing), a code coverage metric can be assigned. Often code coverage is assigned after execution of a test case, especially where the degree to which the source code of a program will be tested by a test case is not known a priori.
For all but the simplest of source code, testing is an iterative procedure. To assist in this iterative process, software developers use code coverage tools, for example, to highlight sections of the source code that have not been executed during testing. Risk of failure can be difficult to ascertain for an untested section. For such sections of the source code, further test case development can uncover issues and/or verify performance and, hence, improve quality.
As described herein, conventional code coverage tools that merely allow a tester to identify untested code do not add significantly to quality control. Further, a need exists for test prioritization techniques that can make testing more efficient.